Repository navigation
fix(inference): show account status and fail over across hosts - #146
Merged
Merged
Conversation
Read CLIProxyAPI account health when refreshing provider routes and show safe per-account status in settings. Reject new routes when no hosted account is eligible.
|
ScriptedAlchemy
marked this pull request as ready for review
November 17, 2026 19:17
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CLIProxyAPI can retain models in its catalog while hosted accounts are rejected or cooling down. T3 also kept one inference host per provider, so a 403, 429, or offline host stopped a remote thread even when another authorized environment could serve the same model.
Show the proxy's per-account availability in Inference accounts and filter unavailable route choices. Connected web and mobile clients discover other shared inference hosts, register them on the executing environment, and refresh discovery as host status changes. The local proxy keeps the requested model and execution environment, probes healthy hosts, retries a bounded request after auth, quota, or transport failures, combines healthy model catalogs, and keeps each thread on its successful host. Existing saved routes migrate to the driver-scoped local proxy path; reconnecting the same primary retains fallbacks. LAN addresses advertised by a host are verified before use.
Local and remote providers now get a stable forwarding address from their first launch. Background discovery respects explicit revocation until the user restores the grant. T3 identifies its own grant rejection in the response. OpenCode and Pi preserve native models, including project models, when a proxy account goes offline.
OpenCode owns retry timing while a turn runs. An explicit service-overloaded response, including HTTP 429, remains a provider error after OpenCode exhausts its retries. On genuine terminal rate-limit failures, the v2 adapter preserves the provider's retry duration and derives a reset time from the event timestamp. The v1 adapter reads the standard retry header when available. This lets the existing per-thread and environment resume controls schedule a continuation. Proxy forwarding handles both seconds and millisecond retry hints.
With the installed OpenCode 2.3.0 CLI and isolated loopback fixtures, both a token-limit 429 and a service-overload 429 produce terminal
provider.rate-limitevents. Their retained response bodies distinguishrate_limit_exceededfromservice_overloaded, and the event includesretryAfterMs. The adapter regression tests cover that distinction.The changed-file gate passed (568 tests, 42 affected package typechecks, format, lint, Knip, GraphQL, and React compilation). The mounted web coordinator test covers primary disconnect, alternative recovery, duplicate status updates and a primary change. Server tests cover scoped grants, revocation, self and LAN routes, proxy model catalogs and native OpenCode/Pi catalog recovery. The historical Claude failures reached the SDK as bodyless 403s across two Studio threads and two models. The retained events do not identify the responding endpoint or account. A real LAN request previously succeeded. In a separate controlled LAN run, T3 relayed the same Claude request after a bodyless 403 to the second host and used that host again for the next request. Deployment and an installed-client fleet cutover remain unverified.
Account affinity is per CLIProxyAPI host. T3 forwards session identity and holds a successful host for a thread, but switching between hosts with multiple eligible accounts cannot guarantee the same account without a shared account-selection contract. Requests above the 8 MiB replay bound stream once through the selected healthy host and cannot retry after a downstream auth or quota response.
Hosted Claude refresh also checks the subscription usage endpoint through CLIProxyAPI for each eligible account. A confirmed exhausted account-wide window supplies a reset time and suppresses the account while the limit is in effect. Slow or denied usage reads leave the account eligible for an inference request. Mixed pools consult account model registrations and cooldowns so another active account can keep its own models available. A live subscription usage read succeeded while two inference models returned 429 with a matching retry delay.
Models: GPT-6.2-Sol and GPT-6.2-Astra. Harness: Codex.